Shape-changing L-sr1 Trust-region Methods
نویسندگان
چکیده
In this article, we propose a method for solving the trust-region subproblem when a limited-memory symmetric rank-one matrix is used in place of the true Hessian matrix. The method takes advantage of two shape-changing norms to decompose the trust-region subproblem into two separate problems, one of which has a closed-form solution and the other one is easy to solve. Sufficient conditions for global solutions to both subproblems are given. The proposed solver makes use of the structure of limited-memory symmetric rank-one matrices to find solutions that satisfy these optimality conditions. Solutions to the trustregion subproblem are computed to high-accuracy even in the so-called “hard case”.
منابع مشابه
On solving L-SR1 trust-region subproblems
In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the ShermanMorrison-Woodbury formula to ...
متن کاملA Riemannian symmetric rank-one trust-region method
The well-known symmetric rank-one trust-region method—where the Hessian approximation is generated by the symmetric rank-one update—is generalized to the problem of minimizing a real-valued function over a d-dimensional Riemannian manifold. The generalization relies on basic differential-geometric concepts, such as tangent spaces, Riemannian metrics, and the Riemannian gradient, as well as on t...
متن کاملL-sr1: a Second Order Optimization Method for Deep Learning
We describe L-SR1, a new second order method to train deep neural networks. Second order methods hold great promise for distributed training of deep networks. Unfortunately, they have not proven practical. Two significant barriers to their success are inappropriate handling of saddle points, and poor conditioning of the Hessian. L-SR1 is a practical second order method that addresses these conc...
متن کاملOn Efficiency of Non-Monotone Adaptive Trust Region and Scaled Trust Region Methods in Solving Nonlinear Systems of Equations
In this paper we run two important methods for solving some well-known problems and make a comparison on their performance and efficiency in solving nonlinear systems of equations. One of these methods is a non-monotone adaptive trust region strategy and another one is a scaled trust region approach. Each of methods showed fast convergence in special problems and slow convergence in other o...
متن کاملSR1—a small RNA with two remarkably conserved functions
SR1 is a dual-function sRNA that acts as a base-pairing regulatory RNA on the ahrC mRNA and as a peptide-encoding mRNA on the gapA operon. The SR1-encoded peptide SR1P binds GapA thereby stabilizing gapA mRNA. Under glycolytic conditions, SR1 transcription is repressed by CcpN and CcpA. A computer-based search identified 23 SR1 homologues in Bacillus, Geobacillus, Anoxybacillus and Brevibacillu...
متن کامل